Payments System Webinar
For 16 years he built payments systems at Square, BlackRock, and Varo. Watch him explain the right and wrong ways to build.
Sign upWe’ve been building for this for three years, said Naveen Molloy, COO of Bitski, a startup company based in San Francisco that has suddenly found itself at the center of the NFT craze.
While most of us were startled to find out that fungible is not a microorganism, and that an NFT can sell for $6.6 million, Bitski had been ready and waiting for the moment. They observed the popularity of digital assets years ago in the virtual world gaming industry. They saw the challenges presented by blockchain transactions. And they built a platform to process blockchain transactions faster.
CockroachDB Engineer Nathan Stilwell interviewed Bitski’s CTO Patrick Tescher and COO Naveen Malloy to discuss NFT basics, Ethereum, and the NFT marketplace, as well as the challenges of processing blockchain transactions.
What is an NFT?
Technically speaking, an NFT (non-fungible token) is an entry on the blockchain that is unique, finite, and distinguishable from every other NFT. Bitcoin, by contrast, is fungible. Meaning that it is indistinguishable from the other coins of its kind. But by now you’ve probably already had this concept explained to you, unpromted, at least 50 times. What’s interesting about this story is not the NFT’s, but the underlying architecture of the marketplace on which they are minted and sold.
When Molloy says that Bitski has been building for this tidal wave of NFT interest for the last three years he means that Bitksi has been building the infrastructure to address the challenge of buying and selling NFTs. A digital asset does not become an NFT until it has been minted on the blockchain. At that point it becomes unique, and can be owned. The challenges in this process are security and speed.
Bitski addresses these challenges with architecture that emphasizes Swiss bank-level security and an innovative approach to processing blockchain transactions.
Bitski’s NFT Marketplace is ‘Shopify for NFTs’
Originally, Bitksi’s intention was to build all the hard infrastructure of a digital wallet as well as an enterprise management layer for the blockchain. This hard infrastructure is what answers the very difficult question of “So you bought an NFT, now where is it?” It’s in a Bitksi wallet. And it’s safe.
Bitski’s expectation was that somebody would come along and build an NFT marketplace on top of Bitksi. But nobody did. There was a clear need for a simple platform to create, sell, and transfer NFTs. So Bitski built a “creator portal” on top of their digital wallet infrastructure. The way they saw it, all the hard work had already been done. The creator portal is just a simple marketplace that allows people to sell their NFTs and use their digital wallets. Exactly the same way that Shopify or Etsy gave people a place to upload images of their work, and then process those transactions. The primary difference between these two endeavors is the blockchain.
The Burden of Blockchain Transactions
There are many existing NFT marketplaces currently. But they are all struggling with the famous challenge of slow blockchain transactions.
It takes at least a minute for any blockchain transaction to go through. This was fine for the original blockchain workloads (like a bank transfer). But if it takes a minute to sell 100 NFTs it will take over an hour to get all those transactions processed. Which creates confusing and unpleasant delays for the end user. On NBA TopShot, the exclusive platform for buying and selling NBA NFTs, you might click to purchase an NFT and then wait 15 minutes or more while the transaction processes. During which time you’re likely assuming that the transaction didn’t work so you need to try again. Or leave.
This is the challenge that Bitksi’s business is built around. They observed the way that the early NFT platforms struggled with transactions. So they embarked to build infrastructure that would speed up blockchain transactions and create better end user experiences. To do this, they built a “Blockchain Transaction Operator” that queues up transactions in batches, gathers all the information that each transaction is going to need, and then broadcasts that batch of transactions to the blockchain.
Instead of sending one transaction each minute to the blockchain, Bitksi broadcasts 1,000 different transactions at once, which allows the transactions to process extremely fast.
If you buy something on Bitski it’s delivered within a minute.
The Architecture of a High Functioning NFT Marketplace
Before building out their existing architecture, Bitski looked at how other companies were trying to solve the problem of blockchain transactions. They saw a lot of solutions using Kafka to get a rapid transaction queue in place but those solutions couldn’t keep track of all the details required in order to make the system work. Bitksi first built their solution using etcd which allowed them to keep track of some of the state for future transactions in a single place. Etcd created it’s own complexities and Bitski didn’t need it anymore when they migrated from Postgres over to CockroachDB.
In CockroachDB they have a couple of indexes designed with everything that blockchain transactions need. Bitski dumps transactions into CockroachDB. In the process Bitski is able to validate whether or not the transactions are going to go through on the database level. So they can create tokens as fast as CockroachDB can create them, and then a task in the background relays created tokens to the blockchain.
Bitski’s NFT marketplace built on CockroachDB has proven to be an order of magnitude faster than what everyone else is doing.
For more detail about Bitski’s NFT marketplace architecture you can skip to this section of their interview with CockroachDB or you can read the formal Bitksi case study.
Navigating traffic & transaction spikes during “Drops”
In the NFT industry, assets are usually released together in what’s known as “a drop.” Buyers are given a notification that certain items will “drop” in 72 hours, or 15 minutes, or some other relatively imminent moment. Once the drop begins, every NFT in the drop will sell out in a matter of minutes or even seconds.
What this means is that hundreds or thousands of transactions are running every second. For most platforms it can take two or three days for them to catch up to all the transactions in those tiny windows. Bitski is all caught up within an hour or two.
In addition to the high blockchain transaction speed that Bitski’s architecture creates, it is also highly consistent and highly available. High availability is essential for Bitski because downtime during a “drop” would mean missed revenue for the NFT creators and a confusing experience for the NFT buyers. This is another reason Bitski chose CockroachDB - the database does not go down. Bitski initially built their application on top of Postgres where they experienced planned and unplanned downtime. Since switching to CockroachDB neither of those have been an issue.
Database Performance Expectations: 100ms Writes
CockroachDB serves as the general purpose database for Bitski. One of the most important workloads that it handles is the login experience. Logins should happen in under one second in order not to lose users before they even get onto the platform.
Around the time of a “drop” there are thousands of queries happening in the login flow. The database needs to deliver low write latency because the entire login experience needs to happen in 100ms or less. 100ms is the cutoff for something to feel ‘instantaneous.’ This is often called the 100ms rule.
Bitski uses Hydra for authentication - which supports CockroachDB natively. Hydra gives Bitski an access token, which gets queried a lot. When Bitski was setting up their second data center they had an issue with read query latency being too slow. They filed a support ticket with the CockroachDBSupport Team and got help setting up a Secondary Index, which lowered the latency from 70-millisecond read queries to 0.5 milliseconds.
The Future of NFTs & Blockchain
This surge of NFT popularity is familiar. It’s impossible not to associate this craze with the original buzz around Bitcoin. Bitcoin endured plenty of doubt and criticism, but has still reached extraordinary value. Will NFTs have the same kind of endurance? At Bistki, they’ve been planning for this for years. And they see the blockchain and NFTs as an inevitable evolution in the way that digital assets are bought and sold. Coinbase will be the platform for currency. And Bitksi will be the platform for NFTs.
One of the primary blockers for this evolution has been the confusion around what blockchain is and how it works. Bitski thinks this blocker will dissolve when people start to go through the experience of making NFT purchases. In the words of Molloy,
Nobody understands how Netflix works, but it doesn’t stop them from enjoying the streaming experience. When you buy a plane ticket you don’t need to calculate how much money is needed for gas for the flight. That shouldn’t be a requirement for transactions on the blockchain either. In a few years blockchain won’t be the topic anymore. People will just buy and sell and trade and not think about the underlying infrastructure.
